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DETAILED ACTION 

1 . This office action is in response to correspondence filed September 1 7, 2008 in 
reference to application 10/674,573. Claims 1-17, and 19-36 are pending in the 
application and have been examined. Please note the examiner of record had changed 
in this case. 

Response to Amendment 

2. The amendment filed September 17, 2008 has been accepted and considered in 
this office action. Claims 1 , 3, 4, 8, 14, and 16 have been amended, and claim 18 has 
been cancelled. 

Response to Arguments 

3. Applicant's arguments, see Remarks, filed September 17, 2008, with respect to 
claims 1-15 have been fully considered and are persuasive. The previous rejection 
under 35 U.S.C. 102 and 103 of claims 1-15 has been withdrawn. 

4. Applicant's arguments with respect to claims 16 and 32 have been considered 
but are moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC §112 

5. The following is a quotation of the first paragraph of 35 U.S.C. 112: 



The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
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art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

6. Claims 1-15 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. 

7. Claim 1 requires limitations that seem to be directed toward subject matter 
described in the specification page 8 line 7 to page 9 line 14. However, the claim 
requires that "data parts from the at least one explicit command based on the acoustic 
word boundaries and using the at least one difference recognition vocabulary." The 
specification seems to define the "explicit command" as being only the first part of the 
utterance, " Add Jack Smith to my phone book;" page 8 line 13-20. On page 9 however, 
the implicit command is used to select a vocabulary used to decode the second part of 
the utterance, which seems to be defined as part of the "implicit command," page 9 
lines 4-14. Therefore there is no support in the specification for using the vocabulary 
selected on the basis of the implicit command for use of decoding data parts contained 
in the explicit command. Therefore claim 1 is rejected as containing new matter. 
Claims 2-15 are rejected as well as they are dependent on claim 1 and do not bring the 
claim in line with what is described in the specification. 
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Claim Rejections - 35 USC § 103 

8. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

9. Claims 16 and 31 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Walker et al. (US Patent 6,434,529) in view of Goldberg et al. (US Patent 6,400,652). 

1 0. As per claim 1 6, 1 7, 1 9, 24- 29, and 31 , Walker teaches a method and program 
storage device readable by machine (Col. 19, lines 25-37), for recognizing at least one 
command and at least one segment of acoustic voice data in a same utterance 
comprising the steps of: 

decoding at least one word in voice data representing the acoustic signal that 
comprises a human utterance and determining the acoustic word boundaries within the 
voice data (Col. 5, lines 49-60 and the example provided in Col. 6, lines 34-48, wherein 
from Col. 5, lines 56-60 it is stated that the recognition result contains the tokens or 
words the user said); 

extracting at least one command from the utterance (Col. 6, lines 36-48, Walker 
extracts the command of "1 want a (hamburgerlburger) with <toppings>" or rule 
<order>, which is determined by previously generating the recognition result identifying 
the words the user said); and 

associating segments in the voice data based on the acoustic word boundaries 
with labels (Col. 6, lines 36-48, Walker clearly presents that the rules or object instances 
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<toppings>,<condiment> , and <veggy> relate to the acoustic data segments identified 
from the user utterance. Other examples are provided in Col. 4, lines 34-40, wherein the 
commands are <play> ,<stop> , and <goto>, and the label <lineno> represents the 
acoustic data segment from the utterance. 

Walker does not specifically teach executing the at least one command utilizing 
undecoded information in the acoustic voice data. 

In the same field of speech recognition, Goldberg teaches executing the at least 
one command utilizing undecoded information in the acoustic voice data (Figure 3B, 
audio (undecoded) is recorded based on the recognition of trigger event (command) 
and stops when terminating event is detected; column 6, line 37-56). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the recording of Goldberg in the system of Walker in order to add 
the ability to record desired information that the user may wish to play back later, such 
as phone numbers, or contact information; column 1 lines 53-58. 

11. As per claim 1 7, Walker teaches the method as recited in claim 1 6, wherein the 
step of extracting includes employing an application, which identifies commands in the 
utterance in accordance with the labels (Col. 4, lines 29-31 and Col. 4, lines 34-45). The 
application program may be referenced directly from scripting language within the tags 
(labels) defined by the rule grammar (Col. 4, lines 29-31). A portion of the rule grammar 
for the example of the media player is shown on Col. 4, lines 34-40, where commands 
such as "play," "go," and "start" are labeled <play>. Also the label is part of the rule 
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grammar for <command> A tags parser program is invoked to interpret the tags in a 
recognition result matching one of the rules, such as Processing of recognition results in 
the application programs may be simplified to an invocation of the tags parser (Col. 4, 
lines 41-45). 

12. As per claim 1 9, Walker teaches the method as recited in claim 1 6, but does not 
specifically mention wherein the step of extracting includes the step of storing at least 
one non-command voice data segment. 

In the same field of speech recognition, Goldberg teaches the step of extracting 
includes the step of storing at least one non-command voice data segment (Figure 3B, 
audio (undecoded) is recorded based on the recognition of trigger event (command) 
and stops when terminating event is detected; column 6, line 37-56). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the recording of Goldberg in the system of Walker in order to add 
the ability to record desired information that the user may wish to play back later, such 
as phone numbers, or contact information; column 1 lines 53-58. 

1 3. As per claim 24, Walker teaches the method as recited in claim 1 6, further 
comprising the step of buffering the utterance to be processed and maintaining the 
utterance in memory during processing of the utterance (Fig. 8 and Col. 14, lines 57-58 
and 62-64). "SUSPENDED" state 136 of the Recognizer (Fig. 8), wherein the 
Recognizer remains in the SUSPENDED state 136 until processing of the result 
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finalization event is completed (Col. 14, lines 57-58). In the SUSPENDED state 136 the 
Recognizer buffers incoming audio. This buffering allows a user to continue speaking 
without speech data being lost (Col. 14, lines 62-64). 

14. As per claim 25, Walker teaches the method as recited in claim 16, wherein the 
step of associating segments includes employing grammars to associate a unique label 
with each command segment in the utterance (Col. 6, lines 36- 44. The association of 
the label to the command segment "I want a (hamburger|burger) with "from the user 
utterance "I want a (hamburgerlburger) with onions and mustard." The labels 
<veggy>and <condiment> are also associated with the words onion and mustard, 
respectively.). 

1 5. As per claim 26, Walker teaches the method as recited in claim 25, wherein the 
label includes a numerical value (Col. 6, lines 36-44, give an example of a user's 
utterance "1 want a burger with onions and mustard," wherein the label "<order> " is 
associated with the acoustic data segment "1 want a (hamburgerlburger) with ."); 

It would have been obvious to a person having ordinary skill in the art to include 
a numerical value to the label. For example, if there was a rule for another "order" such 
as "I want a <flavor> ice cream" the label could have included a number "<order2>"). 

16. As per claim 27, Walker teaches the method as recited in claim 25, wherein the 
grammars include a form for extracting information for an order or verbal contract 
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(Walker et al. teach a system (Fig. 1) that includes result listener 18, parse tree 20, and 
a tags parser 24. The result listener receives the recognition result and uses the 
grammar from grammars 12, which includes the rule that was matched to turn the result 
into a parse tree 20 (Col. 5, lines 61-63), then the tags parser 24 evaluates the parse 
tree 20 and creates an object instance, called a rule object, for each rule it encounters 
in the parse tree 20. The name of a rule object for any given rule is, for purposes of 
example, of the form $name. That is, the name of the rule object is formed by 
prepending a '$' to the name of the rule (Col. 6, lines 14-19). In a specific example, Col. 
6, lines 36-44 describe an example of a form (or rule) for a food order). 

17. As per claim 28, Walker teaches the method as recited in claim 25, wherein the 
grammars include a form for reminding a user to perform a task (Walker et al. teach a 
system (Fig. 1 ) that includes result listener 18, parse tree 20, and a tags parser 24. The 
result listener receives the recognition result and uses the grammar from grammars 12, 
which includes the rule that was matched to turn the result into a parse tree 20 (Col. 5, 
lines 61-63), then the tags parser 24 evaluates the parse tree 20 and creates an object 
instance, called a rule object, for each rule it encounters in the parse tree 20. The name 
of a rule object for any given rule is, for purposes of example, of the form $name. That 
is, the name of the rule object is formed by prepending a '$' to the name of the rule (Col. 
6, lines 14-19). In a specific example, Col. 6, lines 36-44, describe an example of a form 
(or rule) for a food order. It would have been obvious to one having ordinary skill in the 
art that this form or rule could also be applied to remind a user to perform a task). 
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18. As per claim 29, Walker teaches the method as recited in claim 25, wherein the 
grammars include a form for extracting maximum meaningful length segments under 
interruption or silence conditions (Walker et al. teach a system (Fig. 1) that includes 
result listener 18, parse tree 20, and a tags parser 24. The result listener receives the 
recognition result and uses the grammar from grammars 12, which includes the rule that 
was matched to turn the result into a parse tree 20 (Col. 5, lines 61-63), then the tags 
parser 24 evaluates the parse tree 20 and creates an object instance, called a rule 
object, for each rule it encounters in the parse tree 20. The name of a rule object for any 
given rule is, for purposes of example, of the form $name. That is, the name of the rule 
object is formed by prepending a '$' to the name of the rule (Col. 6, lines 14-19). In a 
specific example, Col. 6, lines 36-44, describe an example of a form (or rule) for a food 
order. It would have been obvious to one having ordinary skill in the art that this form or 
rule could also be applied to extract maximum meaningful length segments under 
interruption or silence conditions). 

1 9. Claims 20, 23, and 30 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Walker (US Patent 6,434,529) in view of of Goldberg as applied to 
claim 16 above, and further in view Stammler et al. (US Patent 6,839,670), hereinafter 
Stammler. 
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20. As per claim 20, Walker and Goldberg teaches the method as recited in claim 16, 
but does not specifically mention wherein the step of extracting includes calling a 
vocabulary for recognizing numbers and recognizing the numbers in the utterance. 

However, Stammler does teach wherein the step of extracting includes calling a 
vocabulary for recognizing numbers and recognizing the numbers in the utterance (Col. 
4, lines 59-63). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of calling a vocabulary for recognition of 
numbers and recognizing the numbers in the utterance as taught by Stammler for 
Walker's method because commands requiring storing telephone numbers or changing 
channels require the recognizer to be able to recognize the numbers. 

21 . As per claim 23, Walker and Goldberg teaches the method as recited in claim 16, 
but does not specifically mention wherein the step of associating includes the step of 
changing a recognizer vocabulary and submitting at least one non-command voice data 
segment for recognition. 

However, Stammler does teach wherein the step of associating includes the step 
of changing a recognizer vocabulary and submitting at least one non-command voice 
data segment for recognition (Col. 5, lines 33-41). The speaker dependent recognizer is 
connected without interface to a speaker independent recognizer. In a specific example, 
"call uncle Willi," the word "call" is part of the speaker independent vocabulary and 
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"uncle Willi" is part of the speaker dependent vocabulary (Col. 5, lines 33-41 ), wherein 
"uncle Willi" is the non-command voice data segment. 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of changing a recognizer vocabulary and 
submitting at least one non-command voice data segment for recognition as taught by 
Stammler for Walker's method, because Stammler provides a speech recognition unit 
consisting an independent compound- word recognizer and a speaker dependent 
additional speech recognizer (Col. 2, lines 47-49), wherein the independent recognizer 
recognizes general control command, numbers, names, letters, etc, and the speaker 
dependent recognizer recognizes user-specific/speaker-specific names or functions 
(non-command), which the user/speaker defines and trains (Col. 5, lines 11-13). 

22. As per claim 30, Walker teaches the method as recited in claim 1 6, but does not 
specifically mention wherein the step of determining acoustic word boundaries includes 
finding segment boundaries by iteratively comparing the same utterance to a plurality of 
vocabularies. 

However, Stammler does teach wherein the step of determining acoustic word 
boundaries includes finding segment boundaries by iteratively comparing the same 
utterance to a plurality of vocabularies (Col. 5, lines 38-41, Col. 2, lines 47-49, Col. 4, 
lines 60-63, Col. 5, lines 11-13, and Col. 2, lines 61-65, wherein the step of determining 
acoustic word boundaries includes finding segment boundaries in the speaker 
independent and speaker dependent vocabularies. The speaker independent 



Application/Control Number: 10/674,573 Page 12 

Art Unit: 2626 

recognizer recognizes general control commands, numbers, names, letters, etc., 
without requiring that the speaker or user train one or several of the words ahead of 
time (Col. 4, lines 60-63) and the speaker dependent recognizer recognizes user- 
specific/speaker-specific names or functions, which the user/speaker defines and trains 
(C01.5, lines 11-13). The system permits a speech command input or speech dialog 
control that is for the most part adapted to the natural way of speaking, and an 
extensive vocabulary of admissible commands that is made available to the speaker for 
this (Col. 2, lines 61-65). In a specific example (Col. 5, lines 38-41 ), "call uncle Willi," 
the speaker independent recognizer recognizes "call" and the speaker dependent 
recognizer, "uncle Willi."). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of finding segment boundaries by 
iteratively comparing the same utterance to a plurality of vocabularies as taught by 
Stammler for Walker's method because Stammler provides a system that permits a 
speech command input or speech dialog control that is for the most part adapted to the 
natural way of speaking, and an extensive vocabulary of admissible commands that is 
made available to the speaker for this (Col. 2, lines 60-65). 

23. Claims 21 and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Walker (US Patent 6,434,529) in view of Goldberg as applied to claim 1 , and 
further in view of Kanevsky et al. (US Patent 6,434,520), hereinafter Kanevsky. 
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24. As per claim 21 , Walker teaches the method as recited in claim 1 6, but does not 
specifically mention wherein the step of extracting includes extracting acoustic data 
based on acoustic word boundaries and saving the acoustic data for acoustically 
rendering the acoustic data. 

However, Kanevsky does teach wherein the step of extracting includes extracting 
acoustic data based on acoustic word boundaries and saving the acoustic data for 
acoustically rendering the acoustic data (Fig. 1 and Col. 7, lines 22-30 and Col. 2, lines 
1-4). An audio indexing system and method that includes a speech 
recognition/transcription module 109 (from Fig. 1), which stores the segmented audio 
data stream S1-SN 104 with the corresponding speaker identity tags IDI-ID2 106, the 
environment/channel tags E1-EN 108, and the corresponding transcription T1 -TN 110. 
Each segment may also be stored with its corresponding acoustic waveform, a subset 
of a few seconds of acoustic features, and/or a voiceprint, depending on the application 
and available memory (Col. 7, lines 22-30). Also the user may retrieve stored audio 
segments from the database by formulating queries based on one or more parameters 
corresponding to such indexed information (Col. 2, lines 1-4). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of extracting acoustic data based on 
acoustic word boundaries and saving the acoustic data for acoustically rendering as 
taught by Kanevsky for Walker's method, because Kanevsky provides an audio 
processing system and method for indexing and storing audio data, and an information 
retrieval system which provides immediate access to audio data stored in the archive 
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through a description of the content of an audio recording, the identity of speakers in the 
audio recording, and/or a specification of circumstances surrounding the acquisition of 
the recordings (Col. 1, lines 32-38). 

25. As per claim 22, Walker teaches the method as recited in claim 1 6, but does not 
specifically mention wherein the step of extracting includes extracting acoustic data 
based on acoustic word boundaries and decoding the acoustic data for storage (Fig. 1, 
Col. 6, lines 39-42, and Col. 7, lines 22-30). An audio indexing system and method that 
includes a speech recognition/transcription module 109 (from Fig. 1), which decodes the 
spoken utterances for each segment S1-SN 104 and generates a corresponding 
transcription T1-TN 110 (Col. 6, lines 39-42). The system also stores the segmented 
audio data stream S1-SN 104 with the corresponding speaker identity tags ID--ID2 106 
the environment/channel tags E1-EN 108, and the corresponding transcription T1-TN 
110. Each segment may also be stored with its corresponding acoustic waveform, a 
subset of a few seconds of acoustic features, and/or a voiceprint, depending on the 
application and available memory (Col. 7, lines 22-30). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of extracting acoustic data based on 
acoustic word boundaries and decoding the acoustic data for storage as taught by 
Kanevsky for Walker's method, because Kanevsky provides an audio processing 
system and method for indexing and storing audio data, and an information retrieval 
system which provides immediate access to audio data stored in the archive through a 
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description of the content of an audio recording, the identity of speakers in the audio 
recording, and/or a specification of circumstances surrounding the acquisition of the 
recordings (Col. 1, lines 32-38). 

26. Claims 32-36 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Walker (US Patent 6,434,529), in view of Romero (US 2002/01 1 1803), and further in 
view of Murphy (US Patent 5,544,654). 

27. As per claim 32, Walker teaches a system for recognizing commands and voice 
data in a same utterance comprising: 

an acoustic input, which receives utterances (Fig. 1, audio input 14); and 
a data buffer configured to store audio data representing the utterances (Col. 14, 
lines 62-67, "In the SUSPENDED state 136 (from Fig. 8) the Recognizer buffers 
incoming audio. This buffering allows a user to continue speaking without speech data 
being lost. Once the Recognizer returns to the LISTENING state the buffered audio is 
processed to give the user the perception of real-time processing."); and 

at least one program that executes label-identified commands (Col. 13, lines 8- 

24). 

However, Walker does not specifically mention 

a speech recognition engine configured to match portions of the utterances to 
acoustic models and language models to recognize words and word boundaries in the 
utterance and labels commands in the utterance. 
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Conversely, Romero does teach 

a speech recognition engine configured to match portions of the utterances to 
acoustic models and language models to recognize words and word boundaries in the 
utterance and labels commands in the utterance (Fig. 1, Paragraphs [0028] and 
[0020,0021,0022]. Speech recognizer 100 comprising an acoustic model 104 and a 
language model 116 (From Fig. 1). The recognizer also has a "fast acoustic match" 108, 
which makes use of the acoustic models (from Fig. 1), for comparing a string of 
incoming labels to the items stored in the conceptual vocabulary (Paragraph [0028]). 
Also Romero's paragraphs [0020], [0021], and [0022] show examples of "tags" (or 
labeling) of an utterance, such as in paragraph [0020], for the utterance "Please, give- 
me the phone number of Pedro Romero," the recognizer analyzes the fragment "Give 
me the phone number of as a semantic identifier (command) and tagged "QUERY" or 
"QUERY-EN" and "Pedro Romero" as data and tagged "Pedro_fn Romerojn." 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used the feature of a speech recognizer as taught by 
Romero for Walker's system, because Romero provides a speech recognizer that can 
accept Natural Language utterances as input and directly generate the information 
required to process a user request (Paragraph [0007])). 

However, neither Walker nor Romero specifically mention processing remaining 
portions of the utterance including processing audio data parts separately from the 
commands using a different vocabulary, the vocabulary being selected in accordance 
with at least one command in the utterance. 
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In the same field of speech recognition, Murphy teaches processing remaining 
portions of the utterance including processing audio data parts separately from the 
commands using a different vocabulary, the vocabulary being selected in accordance 
with at least one command in the utterance (figure 10, and column 9 lines 16 - column 
10 line 5, the active vocabulary is determined for each state, and each state is 
determined based on a command.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to limit the active vocabulary to words that are relevant to a particular 
command state as taught by Murphy in the system of Walker and Romero, in order to 
increase accuracy while decreasing the total cost of recognition to the system. 

28. As per claim 33, Walker, as modified above, teaches the system as recited in 
claim 32, wherein the at least one program includes a function which searches the 
utterance for labels output from the speech recognition engine to execute a command 
associated with the label (Walker's Col. 4, lines 41-49, " Processing of recognition 
results in the application program may be simplified to an invocation of the tags parser 
(tags parser program 24) such as "public void interpretResult(RecognitionResult 
recognitionResult { TagsParser.parseResult(recognition Result);}" ). 

29. As per claim 34, Walker, as modified above, teaches the system as recited in 
claim 32, wherein, in accordance with each label, an audio segment is identified and 
processed (Walker's Col. 4, lines 43-49 describe an example of the application program 
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processing a recognition result, wherein the recognition result could be Romero's 
example from Paragraph [0020]) of the tag "QUERY" representing the semantic 
identifier "Give me the phone number of and the tag "Pedro_fn Romerojn" 
representing the data of the utterance "Please, give me the phone number of Pedro 
Romero."). 

It would have been obvious to one having ordinary skill in the art at the time the 
invention was made to have used examples of Natural Language utterances as taught 
by Romero for Walker's system, because Romero provides a speech recognizer that 
can accept Natural Language utterances as input and directly generate the information 
required to process a user request (Paragraph [0007])). 

30. As per claim 35, Walker, as modified above, teaches the system as recited in 
claim 32, wherein the speech recognition engine utilizes grammars with labels, which 
the system uses for assigning labels to decoded commands (Walker's Col. 4, lines 34- 
40, show an example of the rule grammar applied to a media-player application, 
wherein, for example, the system assigns the label to the decoded commands 
(playlgolstart)). 

31 . As per claim 36, Walker, as modified above, teaches the system as recited in 
claim 35, wherein the grammars are represented in Bachus-Naur Form (BNF) (Walker's 
Fig. 4). 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to DOUGLAS C. GODBOLD whose telephone number is 
(571)270-1451 . The examiner can normally be reached on Monday-Thursday 7:00am- 
4:30pm Friday 7:00am-3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached on (571) 272-7603. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

1 1/18/2008 /Talivaldis Ivars Smits/ 

Primary Examiner, Art Unit 2626 
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